i dont love this honestly, but i'm down to release it and see how it performs.

it's a lot of manual work to track if we end up with lots of jobs like these.

wondering if instead when we re-do indexing, we do something more "streaming"-like where we watch for certain things and then run one off queries against individual users. but would have to put more thought into that.

@rickyrombo may have a lot of thoughts here, but I am generally in favor of moving quickly, trying things, and then course correcting if they don't pan out

I don't love it either. There's a lot of downside to not updating scores immediately when the conditions affecting them change. But adding triggers in all the right places also feels fraught. Adding a new feature to the score calculation means making sure we find all the places where that gets changed and triggering a score calculation. We could probably get close enough by leaning on triggers on a bunch of tables. But triggers are also causing us a lot of headache the more we use them 🤷 .

I can spend a little more time and see if I can get the update query to work efficiently with small batches of users so we can just loop on that until its done and throw out all the multi replica logic. Then it's just the same thing we're doing today, only a little slower to finish the cycle so that it doesn't block indexing writes.

Might be a really dumb q: what does a read/calculation of a score look like for a single user? If it's fast enough (or can be made to be fast), maybe we do the calculation and update the score on read, like a cache (or even better yet, no score caching and just compute on demand). That would save a ton of wasted cycles on recalculating scores of inactive users...

Maybe something like:

• If the score hasn't expired, return it
• If the score has soft-expired, return it, and update the score after returning
• If the score is very expired, calculate first, return new result

I'm not generally aware of how the score gets used though or if that's reasonable. If it's being used at the query level for things that kinda falls apart - would need some separate app code probably... but yeah forget all that I say ship as-is. This is probably one of the more gnarly of aggregates as it doesn't have a lane and sort of reaches into all sorts of tables for signals, so of all the ones to be a job I think this one is worthy anyway.....

There are some expensive bits in this query that aren't consistent across users. Play history, reposts, followers all can be huge or small. So maybe the score query returns immediately or maybe it takes a few seconds. And it doesn't help that some of the conditions that invalidate your score are actions taken by other users. We could always set a minimum interval for a score update (only update if updated_at > 5 mins ago or something like that). I think that's an avenue worth exploring for the second pass at this if streaming updates prove to be too complicated to implement in a maintainable/reliable way.

⚠️ GitGuardian has uncovered 4 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}

@raymondjacobson @rickyrombo Latest changes clean it up a bit and moves some of the features we use in the query to be precomputed and updated in a streaming fashion. Not quite ready for on-the-fly usage, but it should be faster!